Non-frontal Model Based Approach to Forensic Face Recognition

Face recognition is a difficult problem and it becomes even more challenging for non-frontal view images (captured by a CCTV camera) commonly encountered in a forensic face recognition case. In a typical forensic case, the surveillance view test image (also called the trace) is captured by a CCTV camera and the reference set consists of frontal view mug shot image of the suspects. Comparison of such disparate views in the test and reference set results in poor recognition performance because most face recognition systems are fine tuned only for comparing frontal or near-frontal view images. The model based approach is often used to ensure that a view based face recognition system gets to compare only frontal view images. This approach involves synthesis of a frontal view image based on reconstructing a 3D face model from the given non-frontal trace. Such a strategy ensures that all the non-frontal view images are transformed to synthesized frontal view image before face comparison. This helps to attain the recognition performance only guaranteed when comparing frontal or near-frontal view images. However, in most forensic cases, the surveillance view trace image is often of very low quality and therefore it is difficult and often not possible to reconstruct the 3D face model and synthesize frontal view image. In this paper, we propose a non-frontal model based approach which ensures that a face recognition system always gets to compare images having similar view (or pose). This requires a virtual suspect reference set that consists of non-frontal suspect images having pose similar to the surveillance view trace image. We apply the 3D model reconstruction followed by image synthesis approach to the frontal view mug shot images in the suspect reference set in order to create such a virtual suspect reference set. This strategy not only ensures a stable 3D face model reconstruction because of the relatively good quality mug shot suspect images but also provides a practical solution for forensic cases where the trace is often of very low quality. For most face recognition algorithms, the relative pose difference between the test and reference image is one of the major causes of severe degradation in recognition performance. Moreover, given appropriate training, comparing a pair of non-frontal images is no more difficult that comparing frontal view images. Due to privacy concerns and data protection laws, it is very difficult to acquire data from real forensic cases. Therefore, to test the proposed non-frontal model based approach, we introduce the PIE ∩ MultiPIE problem: a set of novel and challenging (but fictitious) forensic evaluation cases involving face recognition. Because the PIE and the MultiPIE data set were captured at the same institution in 2000 and 2004 respectively, we formulate a set of fictitious forensic face recognition cases to identify the individuals that are common in these two data sets. These cases simulate some of the challenges faced in a real forensic face recognition case. Our results1 show that at least one commercial face recognition system shows promising recognition performance (Area Under ROC ∼ 76%) for the proposed non-frontal model based approach. We believe that the reported performance figures can be further improved by: (a) using a face recognition system trained specifically for comparing non-frontal view images and, (b) by improving the quality of synthesized images (adding textures, ray tracing, etc).